Generally, it is desirable to automate interactions with electronic documents. One exemplary type of interactive electronic document is a website, where each displayed page of the website may be considered a page of the document. As a person (user, reader, visitor, etc.) interacts with the electronic document, display of the document is modified. That is, user actions cause document reactions. Generally, automation of the user actions is desirable for testing an electronic document, placing an electronic document in a derived state, extracting information from an electronic document, and many other reasons.
A simplistic automated agent can be created to replay user actions. However, these simplistic agents fail if the electronic document changes. For example, a simple approach is to record a series of user actions (e.g., a pointer event such as a mouse click) and a location for each action occurrence (e.g., screen coordinates for the pointer). However, simplistic replay of these recorded user actions fails if the action targets move away from the recorded coordinates. These simple approaches may suffice where the electronic document does not change or is under the control of the party creating the automated agent. A more robust approach is needed for interactive electronic documents that change routinely and/or change outside the control of the party operating the automated agent.